Search for: All records

Past research shows that teachers benefit immensely from reflecting on their classroom practices. At the same time, adaptive and artificially intelligent (AI) tutors are shown to be highly effective for students, especially when teachers are involved in supporting students’ learning. Yet, there is little research on how to support teachers to reflect on their practices around AI tutors. We posit that analytics built on multimodal data from the classroom (e.g., teacher position, student-AI interaction) would be beneficial in providing effective scaffolding and evidence for teachers’ collaborative reflection on human-AI hybrid teaching. To better understand the design opportunities and constraints of a future tool for teacher reflection, we conducted storyboarding sessions with seven in-service teachers. Our analysis revealed that certain modalities (e.g., position v. video) might be more beneficial and less constrained than others in identifying reflection-worthy moments and trends. We discuss teachers’ needs for reflection in classrooms with AI tutors and their boundaries in using multimodal analytics. 

Spatial analytics receive increased attention in educational data mining. A critical issue in stop detection (i.e., the automatic extraction of timestamped and located stops in the movement of individuals) is a lack of validation of stop accuracy to represent phenomena of interest. Next to a radius that an actor does not exceed for a certain duration to establish a stop, this study presents a reproducible procedure to optimize a range parameter for K-12 classrooms where students sitting within a certain vicinity of an inferred stop are tagged as being visited. This extension is motivated by adapting parameters to infer teacher visits (i.e., on-task and off-task conversations between the teacher and one or more students) in an intelligent tutoring system classroom with a dense layout. We evaluate the accuracy of our algorithm and highlight a tradeoff between precision and recall in teacher visit detection, which favors recall. We recommend that future research adjust their parameter search based on stop detection precision thresholds. This adjustment led to better cross-validation accuracy than maximizing parameters for an average of precision and recall (F1 = 0.18 compared to 0.09). As stop sample size shrinks with higher precision cutoffs, thresholds can be informed by ensuring sufficient statistical power in offline analyses. We share avenues for future research to refine our procedure further. Detecting teacher visits may benefit from additional spatial features (e.g., teacher movement trajectory) and can facilitate studying the interplay of teacher behavior and student learning. 

Research indicates that teachers play an active and important role in classrooms with AI tutors. Yet, our scientific understanding of the way teacher practices around AI tutors mediate student learning is far from complete. In this paper, we investigate spatiotemporal factors of student-teacher interactions by analyzing student engagement and learning with an AI tutor ahead of teacher visits (defined as episodes of a teacher being in close physical proximity to a student) and immediately following teacher visits. To conduct such integrated, temporal analysis around the moments when teachers visit students, we collect fine-grained, time-synchronized data on teacher positions in the physical classroom and student interactions with the AI tutor. Our case study in a K12 math classroom with a veteran math teacher provides some indications on factors that might affect a teacher’s decision to allocate their limited classroom time to their students and what effects these interactions have on students. For instance, teacher visits were associated more with students’ in-the-moment behavioral indicators (e.g., idleness) than a broader, static measure of student needs such as low prior knowledge. While teacher visits were often associated with positive changes in student behavior afterward (e.g., decreased idleness), there could be a potential mismatch between students visited by the teacher and who may have needed it more at that time (e.g., students who were disengaged for much longer). Overall, our findings indicate that teacher visits may yield immediate benefits for students but also that it is challenging for teachers to meet all needs - suggesting the need for better tool support. 

Past research shows that teacher noticing matters for student learning, but little is known about the effects of AI-based tools designed to augment teachers’ attention and sensemaking. In this paper, we investigate three multimodal measures of teacher noticing (i.e., gaze, deep dive into learning analytics in a teacher tool, and visits to individual students), gleaned from a mixed reality teacher awareness tool across ten classrooms. Our analysis suggests that of the three noticing measures, deep dive exhibited the largest association with learning gains when adjusting for students’ prior knowledge and tutor interactions. This finding may indicate that teachers identified students most in need based on the deep dive analytics and offered them support. We discuss how these multimodal measures can make the constraints and effects of teacher noticing in human-AI partnered classrooms visible. 

Although students’ self-regulated learning has been studied extensively, past research has not investigated students’ fine-grained, self regulated choice-making processes during learning with visual representations and strategies to support such processes. We conducted design and experimental studies with 148 students to develop and evaluate an intervention package for supporting students’ self-regulated choice-making in using diagrammatic scaffolding in algebra tutoring software. A classroom experiment showed that students with the intervention learned greater conceptual and procedural knowledge in algebra than students in the control condition whose choices were not supported. Also, students with the intervention chose to use diagrams less frequently overall but showed distinctive use patterns that changed over time, indicating a form of self-regulated diagram use. This study demonstrates the importance of understanding and supporting choice behaviors that change over time during learning, going beyond simply measuring the frequency of choice behaviors and encouraging students to engage in these behaviors more frequently. 

Although students’ self-regulated learning has been studied extensively, past research has not investigated students’ fine-grained, self-regulated choice-making processes during learning with visual representations and strategies to support such processes. We conducted design and experimental studies with 148 students to develop and evaluate an intervention package for supporting students’ self-regulated choice-making in using diagrammatic scaffolding in algebra tutoring software. A classroom experiment showed that students with the intervention learned greater conceptual and procedural knowledge in algebra than students in the control condition whose choices were not supported. Also, students with the intervention chose to use diagrams less frequently overall but showed distinctive use patterns that changed over time, indicating a form of self-regulated diagram use. This study demonstrates the importance of understanding and supporting choice behaviors that change over time during learning, going beyond simply measuring the frequency of choice behaviors and encouraging students to engage in these behaviors more frequently. 

In designing learning technology, it is critical that the technology supports both learning and engagement of students. However, achieving both aspects in a single technology design is challenging. We report on the design and evaluation of Gwynnette, intelligent tutoring software for early algebra. Gwynnette was deliberately designed to enhance students’ algebra learning and engagement, integrating several playful interaction and gamification features such as drag-and-drop interactions, an alien character, and sound effects. A virtual classroom experiment with 60 students showed that the system significantly enhanced both engagement and conceptual learning in early algebra, compared to the older version of the same software. Log data analyses gave insights into how the design might have affected the out-comes. This study demonstrates that a deliberate design of learning technology can help students learn and engage well in an unpopular subject such as algebra, a challenging dual goal in designing learning technologies. 

One pedagogical technique that promotes conceptual understanding in mathematics learners is self-explanation integrated with worked examples (e.g.,Rittle-Johnson et al., 2017). In this work, we implemented self-explanations with worked examples (correct and erroneous) in a software-based Intelligent Tutoring System (ITS) for learning algebra. We developed an approach to eliciting self-explanations in which the ITS guided students to select explanations that were conceptually rich in nature. Students who used the ITS with self-explanations scored higher on a posttest that included items tapping both conceptual and procedural knowledge than did students who used a version of the ITS that included only traditional problem-solving practice. This study replicates previous findings that self-explanation and worked examples in an ITS can foster algebra learning (Booth et al., 2013). Further, this study extends prior work to show that guiding students towards conceptual explanations is beneficial. 

Integrating visual representations in an interactive learning activity effectively scaffolds performance and learning. However, it is unclear whether and how sustaining or interleaving visual scaffolding helps learners solve problems efficiently and learn from problem solving. We conducted a classroom study with 63 middle-school students in which we tested whether sustaining or interleaving a particular form of visual scaffolding, called anticipatory diagrammatic self-explanation in an Intelligent Tutoring System, helps students’ learning and performance in the domain of early algebra. Sustaining visual scaffolding during problem solving helped students solve problems efficiently with no negative effects on learning. However, in-depth log data analyses suggest that interleaving visual scaffolding allowed students to practice important skills that may help them in later phases of algebra learning. This paper extends scientific understanding that sustaining visual scaffold does not over-scaffold student learning in the early phase of skill acquisition in algebra.